In the age of multimedia which integrally handles audio, video and other pixel values, existing information media, i.e., newspapers, magazines, televisions, radios, telephones and other means through which information is conveyed to people, have recently come to be included in the scope of multimedia. Generally, multimedia refers to something that is represented by associating not only characters but also graphics, voices, and especially pictures and the like together, but in order to include the aforementioned existing information media in the scope of multimedia, it appears as a prerequisite to represent such information in digital form.
However, when calculating the amount of information contained in each of the aforementioned information media as the amount of digital information, while the amount of information per character is 1˜2 bytes, the amount of information to be required for voice is 64 Kbits or over per second (telephone quality), and 100 Mbits or over per second for moving pictures (current television reception quality), and thus it is not realistic for the aforementioned information media to handle such an enormous amount of information as it is in digital form. For example, although videophones are already in the actual use via Integrated Services Digital Network (ISDN) which offers a transmission speed of 64 Kbps/s˜1.5 Mbps/s, it is not practical to transmit video shot by television cameras directly through ISDN.
Against this backdrop, information compression techniques have become necessary, and moving picture compression techniques compliant with H.261 and H.263 standards suggested by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) are employed for videophones, for example. Moreover, according to information compression techniques compliant with the MPEG-1 standard, it is possible to store picture information in an ordinary music CD (compact disc) together with sound information.
Here, MPEG (Moving Picture Experts Group) is an international standard on digital compression of moving picture signals, and MPEG-1 is a standard for compressing moving picture signals, namely, television signal information, approximately into one hundredth. Furthermore, since MPEG-1 targets for moderate picture quality which can be realized by a transmission speed of about 1.5 Mbps, MPEG-2, which was standardized with a view to satisfy requirements for further improved picture quality, allows data transmission of moving picture signals at a rate of 2˜15 Mbps.
Under the present situation, MPEG-4 that achieves a higher compression ratio than that of MPEG-1 and MPEG-2, has been standardized by the working group (ISO/IEC JTC1/SC29/WG11) which has been engaged in the standardization of MPEG-1 and MPEG-2. A strong error-proof technique has initially been introduced to MPEG-4 for not only allowing efficient coding at a low bit rate but also allowing reduced subjective deterioration of image quality even if a transmission line error occurs. Furthermore, ISO/IEC and ITU-T (Joint Video Team (JVT)) are now jointly making efforts to standardize ITU H.264|AVC as a next-generation picture coding method, and Study of FCD (SoFCD) is the latest standard as of this point in time (See “Study of Final Committee Draft of Joint Video Specification”, Joint Video Team (JVT) of ISO/IEC and ITU-T VCEG, JVT-F100, 15 Feb., 2003).
In H.264|AVC, differently from the conventional moving picture coding, it is possible to select an arbitrary picture as a forward reference picture from among a plurality of pictures. Here, a picture indicates a frame or a field.
Reference pictures and the like are stored in a memory.
A memory includes a short-term memory and a long-term memory. A short-term memory stores a plurality of pictures which have been decoded immediately previously, and more specifically, it stores reference pictures such as P pictures (forward predictive coded pictures) and B pictures (bi-predictive coded pictures) in term of MPEG-1 and MPEG-2. A long-term memory is used for storing pictures for a longer time than the short-term memory.
A short-term memory is usually a FIFO (First-In First-Out) memory, and in order to store a new picture into the short-term memory whose storage area is full of pictures, the oldest picture among the pictures which have been stored in the short-term memory is removed from the memory and the new picture is stored in that free area after the removal. Therefore, in the case where it is necessary to refer to, for a long time, the reference picture which is to be removed from the short-term memory, the reference picture is usually moved in advance from the short-term memory to the long-term memory for storage. As a result, it becomes possible to refer to the reference picture for a long time. The long-term memory stores the picture in a specified area. Therefore, the picture stored in that area can be referred to unless the same area is designated and overwritten.
Furthermore, decoded pictures are stored in a display memory temporarily before being displayed. This display memory has a FIFO structure in which the pictures are overwritten in display order from the older one.
Sophisticated memory management is required for using such a memory efficiently, so a mechanism for managing a memory is incorporated into H.264|AVC.
Memory management commands used for the above management are as follows, for example:
1. Command to select pictures which can be referred to;
2. Command to release a memory area of a short-term memory where a picture unnecessary as a reference picture for predictive coding is stored;
3. Command to move the picture stored in the short-term memory to the long-term memory; and
4. Command to release the memory areas for all the pictures and initialize the information concerning picture decoding.
In picture coding and decoding, a signal that indicates the reference picture (reference picture indication signal) is necessary for selecting, on a block-by-block basis, a picture with a smaller prediction error as a reference picture from among the pictures which can be referred to. It is possible to narrow down the number of candidate reference pictures to an appropriate value by selecting in advance the pictures which can be referred to, and thus to save the number of bits of the reference picture indication signal required for each block.
In the case where the picture is moved from the short-term memory to the long-term memory, the picture stored in the short-term memory is removed because it is useless duplication of the same picture which is stored in both the short-term memory and the long-term memory.
In the above picture coding method and picture decoding method, the picture coding apparatus codes the memory management command that instructs removal of an unnecessary picture from a memory and the memory management command that instructs moving of a picture from the short-term memory to the long-term memory, outputs them, and transmits them to the picture decoding apparatus, which decodes them. These memory management commands are added to the coded pictures for transmission. In the case where the picture added with the memory management command is lost due to a transmission error or the like, the ordering of the pictures in the memory cannot be properly reconstructed on the part of the picture decoding apparatus, and thus the pictures cannot be decoded.
In consideration of this problem, it is conceivable to transmit the memory management command (MMCO) again. This retransmission of the memory management command allows proper reconstruction of picture ordering in the memory by the retransmitted memory management command even if the picture added with the memory management command is lost due to a transmission error or the like.
Picture numbers are assigned to respective pictures in order to indicate pictures to which memory management commands are applied and pictures which have been added with memory management commands in the case where the commands are to be retransmitted. These picture numbers indicate the pictures to be managed and the pictures which were initially added with the memory management commands.
Picture numbers are the numbers assigned in ascending order (coding order) in a coded picture stream, and also used for deriving the display order of pictures in the display memory. It should be noted that the picture numbers are coded as remainders of a predetermined number MaxFrameNum because the picture number becomes significantly larger as the number of pictures increases.
On the other hand, an IDR picture is introduced in order to ensure that even if the picture rate or the like of a coded picture stream is switched in the middle of the stream, the subsequent coded picture stream can be properly decoded. This IDR picture is a picture for prohibiting reference to the pictures which are earlier than the IDR picture, and the memory areas for the pictures are released and the information concerning the picture decoding is initialized at the point in time when this IDR picture is coded. Therefore, the pictures stored in the memory after the coding or decoding of the IDR picture are different from the pictures stored before that. Furthermore, an initialization command, instead of the IDR picture, is sometimes used. The IDR picture initializes the decoding information of all the pictures including the picture numbers, while the initialization command initializes only a part of the information (such as the picture numbers).
In addition, in the case where the coded picture stream is edited, namely, in the case where another coded picture stream is newly generated by extracting a part of each of a plurality of coded picture streams and combining them, inconsistency of the picture numbers or the like occurs in the combined point (editing point).
However, there is the following problem in the above picture coding method and picture decoding method: even if the retransmitted memory management command indicates the picture to be managed using the picture number, it is unknown, on the part of the picture decoding apparatus which has received the retransmitted memory management command, to which picture the memory management command is to be applied, and thus a malfunction is induced.
In other words, since the picture numbers are represented as remainders of a predetermined number and the picture numbers are initialized to 0 by the IDR picture and the initialization command, the same picture number is assigned to different pictures. Therefore, it becomes difficult to specify one picture by the picture number and thus the malfunction is induced.
For example, the display memory allocates a new area required for display of respective pictures by removing the pictures in display order from the earlier one based on the information concerning their display order. Once the picture numbers subsequent to the IDR picture or the initialization command are reset to 0, the information indicating the display order is also initialized. As a result, even if the picture coding apparatus retransmits the memory management command for management of the picture earlier than the IDR picture or the like, a malfunction occurs in the picture decoding apparatus which has received the command because there exists no picture to which the memory management command is to be applied.
The present invention is conceived in view of this problem, and the object thereof is to provide the picture coding method and the picture decoding method for preventing occurrence of a malfunction caused by retransmission of a command.